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ABSTRACT 



Technology permeates the lives of most Americans: voice 



mail, personal computers, and the ever-blinking VCR clock have become 
commonplace. In schools, it is creating educational opportunities at a 
dizzying pace and, within and beyond the classroom, it is providing 
unprecedented access to a universe of ideas and resources. As a next step, 
the education community can harness the same power technology has to enrich 
what and how people learn to support the development and dissemination of 
expectations for students' learning in science and mathematics. By 
capitalizing on the burgeoning opportunities for students to explore the 
stimulating environments afforded by innovative technologies, the processes 
by which students acquire, apply, and extend their scientific and 
mathematical powers can be more fully examined. This paper describes some of 
the ways in which technology can: (1) support the ongoing dialogue to decide 

what performance standards should and can be established; (2) support 
understanding of established standards; (3) provide models and guidelines for 
developing and using standards; and (4) offer an ever- expanding repertoire of 
capabilities that will enlarge the conception of challenging performance 
standards for science and mathematics. Supporting dialogue about performance 
standards is discussed by focusing on computer networks and virtual 
environments. Supporting an understanding of performance standards, providing 
models and guidelines for developing and using performance standards, and 
enlarging the conception of challenging performance standards are also 
discussed. (Contains 19 references.) (ASK) 
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The Role of Technology in Advancing Performance Standards 
in Science and Mathematics Learning 



Edys Quellmalz 

Technology permeates the lives of most Americans: voice mail, personal computers, and the ever-blinking VCR clock 
have become commonplace. In schools, it is creating educational opportunities at a dizzying pace and, within and 
beyond the classroom, it is providing unprecedented access to a universe of ideas and resources. As a next step, the 
education community can harness the same power technology has to enrich what and how people learn to support 
the development and dissemination of expectations for students’ learning in science and mathematics. By 
capitalizing on the burgeoning opportunities for students to explore the stimulating environments afforded by 
innovative technologies, we can examine much more fully the processes by which students acquire, apply, and extend 
their scientific and mathematical powers. 

In this chapter, I describe some of the ways in which technology can (1) support the ongoing dialogue to decide what 
performance standards should and can be established, (2) support understanding of established standards, (3) provide 
models and guidelines for developing and using standards, and (4) offer an ever-expanding repertoire of capabilities 
that will enlarge our conception of challenging performance standards for science and mathematics. My suggestions 
about the role that technology can play in the development of performance standards reflect my view that standards 
are malleable over time and that efforts to establish and achieve them are dynamic, ongoing, and recursive. 

Supporting Dialogue About Performance Standards 

An immediate contribution that technology can make to standards-setting efforts is to support dialogue about 
standards as they are being proposed and employed. Communication technologies like e-mail, teleconferencing, 
bulletin boards, and chat rooms can host electronic meetings, deliberations, and debates about what student 
performance levels should be in science and mathematics. Technology enables dynamic interactions among 
stakeholders during the process of considering and setting performance standards, as well as in sessions arranged to 
judge how performance standards are exemplified in student responses. It can extend standards-setting conversations 
in terms of numbers of participants, the interests and expertise they represent, and their locations. Educators from 
varying levels of the educational system, from elementary to higher education, can participate. Mathematicians, 
scientists, researchers, and developers can log on and "chime in." Business and other local community members can 
also enter the debate. 



Computer Networks 

Computer networks can involve stakeholders from geographically distributed and remote sites. In an analysis of the 
impacts of the National Science Foundations ’ (http://www.nsf.gov) Networking Infrastructure for Education (NIE) 
program, Kozma and Quellmalz (1996) found that the development of partnerships and infrastructures to support 
the development of such outreach is well underway. NIE has provided support for national testbeds such as the CoVis 
Project (http://www.covis.nwu.edu) and the National School Network Testbed (http://nsn.bbn.com). Statewide efforts 
linking schools, communities, businesses, science centers, laboratories, museums, and libraries to achieve science and 
mathematics education reform include projects such as Network Montana (http://nmp.umt.edu/), the Vermont 
Technology Alliance (http://www.ra.terc.edu/alliance/TEMPLATE/state_connections/VT/vt_state.cfm), Alaska’s 
Coming into the Country Project (http://red.www.nsf.gov/EHR/RED/NIE/awards/a9454193.htm), Missouri Supporting 
Teachers (MOST), and the New Jersey Networking Infrastructure for Education Testbed. Local electronic 
educational communities are also being nurtured by computer networks in projects such as Union City On-Line 
/http://www.iinion-citv.kl2.ni.us~). Pittsburgh Common Knowledge (http://info.pps.pgh.pa.us), the Blacksburg 
Electronic Village (http://www.bev.org), and the Native American Mathematics and Science Technology project 
(http://red.www.nsf.gov/EHR/RED/NIE/awards/a9554472.htm). Moreover, projects such as the Math Forum 
(http://forum.swarthmore.edu/) and Geometry Forum (http://www.geom.umn.edu/docs/forum/) have been developed to 
promote reform goals by enhancing Web-based communication and collaboration among teachers, researchers, and 
students and by making rich, interesting mathematics resources available. The various forms of participation and 
communication afforded by these networks could be used as avenues for structuring formal and informal 
interactions about science and mathematics performance levels. 



Virtual Environments 

Multi-user virtual environments (i.e., electronic communities) offer another promising venue for on-line, real-time 
dialogue about performance standards. Researchers at SRI International, for example, have created a virtual 
community, the Teacher Professional Development Institute (TAPPED IN) 

(http://www.tappedin.sri.com/info/partners.html/), to help educators understand the benefits afforded by emerging 
Internet technologies t Schlager & Schank. 1997) . TAPPED IN creates a shared virtual place, based on the model of a 
^ ference center, where teachers can meet to learn from one another, be exposed to a variety of education reform 



concepts and approaches, find high-quality resources, and contribute some that they have found useful. Discussions 
about performance standards for mathematics and science learning could be readily incorporated into the 
professional development activities hosted within TAPPED IN by reform project members from the Math Forum, the 
Geological Society of America’s Earth (http://www.tappedin.sri.com/info/partners.html), and the Lawrence Hall of 
Science’s Great Explorations in Math and Science (GEMS) (http://www.lhs.berkeley.edu/GEMS/GEMS.html/). Within 
TAPPED IN, groups and individuals could propose and debate performance levels grounded in shared displays of 
students’ responses. Multimedia files, including digitized movies, can provide access to modes of student 
performance previously accessible only through classroom observation. Electronic bulletin boards and permanent 
white boards can archive comments and suggestions. These technologies can permit conversations about documents 
such as student benchmarks or data summarizing how different groups of students fare on assessment tasks. 

In classrooms where teachers must integrate performance standards into instructional activities, Internet tools could 
allow them to transfer and share with their remote colleagues files of students’ science or mathematics investigations. 
Electronic communication systems can connect teachers as collaborators not only with other educators, but with 
scientists and mathematicians who can contribute their perspectives to deliberations about the quality of students’ 
work and how well that work meets performance standards. 

Students, too, can participate in conversations about performance standards by taking hold of the judgment process 
and learning to monitor their own progress. For example, in the Computer As Learning Partner Project, students 
integrate evidence that they gather for a science investigation and formulate arguments in support of rival theories 
(Linn. 1991) . The introduction of an agreement bar at the bottom of the electronic laboratory notebook allows 
individuals to indicate their concurrence with the conclusion recorded by the group. Linn reports that these 
agreement bars have led to interesting discussions, challenges, and explanations of alternative conclusions. Tools such 
as the agreement bar could also be used as students monitor the extent to which their responses meet established 
criteria and performance standards. More recently, Linn (1997) has developed a Web-based discussion tool called 
SpeakEasy that supplies a networked database of science information and tools for organizing evidence and 
constructing arguments. This project component could also host discussions about performance quality. 

The educational technology projects created by the Cognition and Technology Group at Vanderbilt (CTGV) (1997) 
offer other models of how technology can support conversations about performance standards in mathematics and 
science. In the Adventures of Jasper Woodbury series and the Stones River Mystery (two series developed by CTGV) 
the program displays representative responses of students to embedded assessment challenges. Students are then 
asked to decide how, for example, their choice of a data collection instrument or pollution clean-up plan relates to 
the choices made by a representative sample of students. This strategy for stimulating students to compare their work 
to the responses of other students could also be used to engage them in appraising how their work meets 
performance standards. 



Supporting an Understanding of Performance Standards 

Information technologies can present varied types of information about performance standards to a range of users 
and audiences. Dissemination methods may be as straightforward as on-line reports that describe and explain 
standards. And, although technology can store and transmit information about standards, on-line libraries and 
resource repositories can provide even more support for stakeholders who are seeking to understand, use, adapt, 
extend, or challenge performance standards. 

Information explaining performance standards must integrate agreed upon goals for science and mathematics 
learning with curriculum standards at particular grade levels, appropriate methods for demonstrating achievement, 
and judgments about how well students’ work meets the standards. One valuable role technology can play is to 
disseminate the standards set by national, state, or local agencies. Electronic and CD-ROM-based versions of science 
and mathematics standards can provide alternative vehicles for conveying the consensus about standards for 
mathematics and science learning reached by various groups. 

The capacity of computers to store and organize vast amounts of information permits purposeful, efficient searches. 
Relational databases can be employed that render voluminous, detailed standards collections easily searchable 
through point and click browsing options. Thus, a fifth-grade teacher could access an on-line collection of 
mathematics or science standards and easily navigate the collection to locate standards relevant for her students. 

Additional explanatory material to promote understanding of performance standards for mathematics and science 
learning has oftentimes been published by national or statewide assessment programs. Testing programs have 
distributed "samplers" consisting of illustrative test items and performance tasks, scoring rubrics, and examples of 
students’ work representing performance levels. On-line or CD-ROM versions of such samplers can be developed for 
a range of audiences and used to define, explain, and exemplify the performance standards set for mathematics and 
science tests. 

4° a case in point, a project developed at SRI International to support assessment and professional development 



programs has a number of features that can support standards-setting efforts. The project, Performance Assessment 
Links in Science (PALS) (http://www.tappedin.sri.com/pals/), is a prototype of an on-line resource bank of 
standards-based performance assessment resources in science (Ouellmalz & Schank, 1997) . Supported by a planning 
grant from the National Science Foundation, the prototype links a set of science performance assessment tasks 
developed by the Council of Chief State School Officers State Collaborative for Assessment of Student Standards 
(SCASS) and the California Systemic Initiative Assessment Consortium (CSIAC) to the National Science Education 
Standards (NSES) and the SCASS Science Framework. The on-line version of each science task includes its 
administration procedures, student booklet, scoring rubric, sample student work, and technical quality data from field 
tests of the tasks. Only tasks with field test data meeting technical quality review standards are included in the 
resource bank. 

In the PALS electronic library, the samples of students’ work serve as benchmarks for performance levels and 
standards. Furthermore, the technical quality data displaying the distribution of scores received by the field test 
population provide evidence about the potential implications of performance standards set at varying levels. Such 
resource libraries can provide user-friendly access to organized collections of curriculum standards, content 
standards, performance standards, assessment tasks, scoring rubrics, and samples of students’ work representing the 
range of performance levels. 

Providing Models and Guidelines for Developing and Using Performance Standards 

Another contribution that technology can make to standards’ setting efforts is to provide examples and procedures 
developed by standards-setting groups that other stakeholders in the educational community can use or build upon. 

It can present models and guidelines for applying performance levels. Guidelines and examples may be developed 
for use as highly structured rater training sessions, demonstrations to educational audiences, or informational 
scenarios for broader audiences. Computer programs can automatically analyzeandmonitor scoring data and certify 
applications of performance standards during on-line or CD-ROM-based scoring sessions to provide feedback on the 
accuracy and reliability of scoring and/or classifications of learners’ responses at levels such as proficient or 
advanced. 

In addition, technology can demonstrate the process of determining if student work meets performance standards. 
Systems can guide users through the process as would an expert in a demonstration. For general audiences in the 
educational, business, and lay communities, technology can also simulate the process of applying performance 
standards by presenting brief scenarios and examples. 

Alignment of Standards with Assessment 

The PALS electronic library design and resources provide models and guidelines that can be used for setting and 
applying performance standards. The PALS system can support professional development aimed at aligning 
standards with investigations designed by groups of curriculum and assessment specialists to allow students to show 
what they know and can do with regard to standards. The PALS design permits users to link their state or local 
standards to national science and mathematics standards. In PALS, a coding scheme was developed for the NSES and 
for the SCASS framework. State and local education agencies can create coding schemes for their science and 
mathematics standards and cross index them to the NSES or National Council of Teachers of Mathematics standards. 

In addition to providing examples and guidelines for the alignment of state and local standards to national science 
education standards, the PALS prototype created a set of criteria and procedures for judging the extent to which 
assessment tasks designed to test the national science standards require the science knowledge and inquiry processes 
presented in the standards. Examples of such alignment criteria and procedures can also be used to involve educators 
and other stakeholders in the process of determining the appropriateness of various assessments. Given the pervasive 
mismatch of many currently available tests of science and mathematics learning with reform-based frameworks and 
standards, alignment sessions and technology to support them are sorely needed. 

PALS further supports understanding of the relationship among components of standards-based assessment and 
standards. PALS offers a search and select function to support assessment planning. When the user selects science 
standards she or he wants to test, a list of science performance tasks in the prototype bank that are designed to test the 
selected standards is generated by the system. The PALS technology database design allows users to initiate their 
search by selecting local or national standards. 

The user can then browse the model tasks designed to test the selected standards, the scoring rubrics, examples of 
students’ work, and technical quality data. Assessment personnel and teachers might decide to use the model 
performance tasks in the resource bank as they are provided, adapt the tasks and/or rubrics, or reference the tasks as 
models or ideas for other investigations. These features of PALS shepherd users through the process of moving from 
broad curriculum standards to activities that test them. Moreover, the scoring rubrics and benchmark papers in the 
resource bank provide real examples of what student performance looks like for each standards-based task and how 
,u ~ scoring rubric is used to judge students’ work at different quality levels. Score distributions from the field test 



can provide some empirical data for estimating how other students might meet these performance standards. 

Upon selection of a final task or set of tasks, an assessment planning chart is generated by the PALS system that 
displays the extent to which the tasks cover those science standards. Therefore, educators can "see" the coverage of 
standards afforded by a set of assessment items and tasks. These features of the PALS on-line system architecture are 
designed to support educators by providing models and guidelines for aligning standards with assessment tasks, 
scoring rubrics, and student benchmarks and score distributions. 

Training on the Use of Performance Standards 

Another major function the PALS system offers is on-line rater training. Raters from geographically distributed 
locations participate in on-line training and calibration to learn to use scoring rubrics reliably. The training system 
presents scoring rubrics, benchmark models, practice sets, and calibration rounds ( Ouellmalz, 1984) . A trainee’s 
scores are automatically saved by the system and compared to benchmark ratings, so that agreement levels can be 
calculated and raters can be referred to additional practice or certified to proceed to independent scoring. Such rater 
training packets can be used as professional development experiences in how to judge student work using explicit 
criteria representing established performance standards. Furthermore, such technology-based rater training 
procedures could be used by standards-setting bodies as they analyze student work, apply rating criteria, and 
establish the performance standards. 

Enlarging Our Conception of Challenging Performance Standards 

It is widely held that the use of multiple-choice tests has severely narrowed the lens through which we can view 
science and mathematics learning. Views of what we can and should expect from schooling have been inextricably 
intertwined with the limited aspects of science and mathematics learning we could examine within traditional test 
formats. For many years, the medium became the message— tests of facts and terminology drove the curriculum. 
Although economics and logistics have most often been blamed for the predominance of multiple-choice testing, 
until recently there were few alternative assessment methods that met technical quality standards for reliability, 
validity, generalizability, and fairness. Now, however, technologies offer more tools we can employ to elicit, collect, 
document, analyze, appraise, and display kinds of student performances that have not been readily accessible through 
traditional testing methods. They are offering ways to breach the gap between standards, instruction, and assessment, 
and to achieve the seamless integration of assessment with instruction. Technology allows the education community 
to appraise performance by employing a greater variety of innovative methods. 

Expanding Measurable Outcomes 

The capacities of media, computers, and networks can permit mathematics and science educators to broaden and 
deepen expectations for student learning. Evidence of science and mathematics learning need no longer be limited to 
questions of factual knowledge and standard problems presented in timed, paper-and-pencil tests. Technology now 
allows educators to set standards not only for subject matter knowledge and inquiry, but for communication, 
metacognition, and collaboration (' Ouellmalz & Hoskvn. 1997) . These cognitive and social skills go beyond the ones 
typically measured by traditional testing methods. Performance standards can be set for breadth, depth, and 
appropriateness of mathematics and science concepts as well as for knowledge and integration. Technology can also 
allow more comprehensive performance standards to be set for inquiry skills related to problem analysis, planning, 
strategy implementation and revision (including the use of tools), solutions, alternative interpretations, and transfer to 
novel situations. Metacognitive skills related to planning, monitoring, and self evaluation can be nurtured and 
documented and appraised in terms of performance standards. Technology also allows appraisals of the effectiveness 
of mathematical and scientific communications and presentations that vary in aim and form. These can be inspected 
for standards related to clarity, focus, coherence, use of acceptable mathematical and scientific evidence, reasoning, 
terminology, and integration of media and representation forms. Group collaboration, heretofore accessible 
primarily through classroom observations, can be encouraged and documented by various media and supported by 
technological storage and analysis capabilities. 

Expanding Available Methods for Designing Assessments 

A significant contribution of technology to standards-setting is the potential for the design of complex, authentic 
tasks that require the use of conceptual understanding, reasoning, communication, and collaboration called for in 
reform standards. Technologies enable the design of student assessments that test the range of cognitive, 
metacognitive, and social strategies elaborated in national standards documents for science and mathematics learning. 
Technologies can expand expectations of what students can do by providing rich environments and tools for 
investigation. With new technologies, challenging assessments can be administered; many types of student responses 
collected; and scoring and data analysis guided or, for some types of responses, accomplished fully. Moreover, they 
can support pilot modeling studies of the implications of standards for different populations of students. 

g - iched task environments. To the extent that assessment tasks permit challenging investigations, or fail to do so, 



performance standards and levels are tied to the nature of the tasks. As Glaser and Baxter (1997 ) have pointed out, 
performance assessments vary considerably in the extent to which they call for rich knowledge structures and deep 
cognitive processing. Questions asking for identification of the parts of a butterfly or the solution to a two-step 
mathematics problem set quite different boundaries from tasks asking for analyses and explanations of the 
interdependencies and effects of a change in conditions on the ecology of a watershed, or the best estimates of 
clean-up costs for a toxic waste spill. 

In our analyses of the Networking Infrastructure for Education program described earlier, there are many leading 
edge network-based programs that provide models for the ways that technology can expand the horizons of 
mathematics and science learning, assessment, and performance standards. We also share examples from NSF’s 
Advanced Applications of Technology program to illustrate how next generation technology will transform the 
field’s expectations for learning of science and mathematics. 

Complex, authentic tasks allow students to show what they know and can do in interesting and challenging 
investigations and open the possibilities for students to engage in investigations that are real, complex, and beyond 
typical classroom resources. Students have access to a wide range of physical phenomena through Internet 
technologies, microworlds, simulations, and microcomputer-based laboratories. Digital encyclopedias and libraries 
allow students to access vast resources as they plan and execute projects. In addition, students can seek input from 
experts and receive guidance from telementors. 

Technologies make available to students the investigative tools used by scientists and educators. These tools provide 
access to processes that are not directly observable, are too expensive or dangerous, or that take place too quickly or 
too slowly or on a scale that is too small or too large ( Horwitz, 1996 ; Forbus, 1996) . Scaffolding in varying forms 
helps students to navigate resources, take advantage of rich environments, and use investigative and analytical tools. 

For example, the Science Learning Network (http://www.fi.edu/sln/ connects kindergarten-through-eighth-grade 
schools with the vast resources of museums and industry through UniVERSE (http://www.fi.edu/sln), an on-line 
electronic librarian. In WhaleNet (http://whale.wheelock.edu), students are connected to scientists/naturalists and 
research data about the marine ecosystem so that they can engage in real-time data collection about issues such as 
marine mammal migration patterns and pollution. A computer-based exploratory tool called GenScope allows 
middle school, high school, and college students to explore visual, interactive, causal connections and interactions 
among biological processes at levels involving molecules, chromosomes, cells, organisms, pedigrees, and populations 
(Horwitz. 1996) . In the HyperBio Project, a hypermedia authoring system uses the full range of digital, audio, and 
video technologies available in multimedia personal computers to illustrate powerful biological principles such as 
homeostasis and natural selection across a wide range of phenomena ranging from the molecular to the ecosystem 
levels (Jacobson. 1996) . In the Science Theater/T eatro de Ciencia Project a modeling tool is being used to give 
elementary school students a medium for expressing and exploring ideas about how things work, for example, how 
tumors form, what makes rainbows, or how predators hunt their prey ( Lewis, 1996) . 

A number of projects are transforming scientific visualization techniques and tools used by scientists to support 
students’ robust understanding of geoscience topics such as weather, climate, and atmospheric green house effects. 
These projects enable students to examine data sets used by the scientific community, create their own data, and draw 
upon models. In the CoVis Project, high school students conduct project-based scientific inquiry in atmospheric 
science such as daily weather and long-term climate (Pea & Gomez, 1996) ; teleconferencing, desktop video, and 
visualizations technologies support students as they record, share, and critique their inquiries in a Collaboratory 
Notebook built from a hypermedia database. In a related project, Visualizing Earth, weather and plate tectonics are 
studied by students using the power of remote sensing and Geographical Information Systems (Barstow, 1996). The 
Global Learning Observations to Benefit the Environment (GLOBE) program 

(http://globe.fsl. noaa.gov/fsl/html/aboutglobe. cgi?intro&/) combines local field study with global investigations and 
technology tools by teaching kindergarten-through-twelfth-grade students to collect environmentally significant 
measurements around their school site (e.g., weather, hydrology, soils, land cover), send their measurements over the 
Internet to databases in research laboratories, and use visualization tools and worldwide databases to make 
comparisons and investigate patterns. The Science Learning in Context project focuses on the applications of 
portable, networked, hand-held computers in student field projects (Tinker, 1996) . The project brings technologies 
outside the lab to support explorations in rich natural settings. 

The SimCalc Project (http://www.simcalc.umassd.edu/simcalc/) challenges deeply institutionalized expectations about 
how mathematics can be learned, by whom, and when (Kaput & Roschelle, 1996) . The project employs visualization 
tools and real-time data acquisition. Students in grades three to thirteen apply basic principles of calculus related to 
rate, approximation, and accumulation of variable quantities in simulations such as observing ducks in a pond, 
watching marchers in a band, or driving a vehicle. 

These innovative computer-based environments can serve as resources for the design of assessment tasks that can 
provide rich evidence about student learning in mathematics and science. The environments allow deep and extended 
^ — mination of students’ progress in mathematics and science subject matter understanding, inquiry strategies, 



metacognition, communication, and collaboration. 

Alternative modalities of administration, representation, and response. Technologies offer opportunities for 
administering assessments in standard, but varied modalities, to include accommodations for students with disabilities. 
Students with different learning styles can conduct investigations in which they access resources through multiple 
media, use scaffolded investigation tools, display their findings in graphic and visual forms, and create presentations 
and reports in a variety of media (e.g., text, graphics, audio, video). Similarly, these media will offer more 
opportunities for students from diverse linguistic backgrounds to engage in and respond to assessment tasks that are 
not text-bound. Projects employing innovative computer-based environments report promising impacts on students 
who perform poorly on traditional text-based assessments. The Cognition and Technology Group at Vanderbilt 
University has found that their video and interactive technologies with dynamic, visual, and spatial representations of 
events allows students to more easily form rich mental models of problems. Teachers using the series report that their 
students who had difficulty reading were able to contribute to solutions in the Jasper Woodbury mysteries and gain 
new respect from their peers ( CTGV, 1997) . Therefore, technology can increasingly improve opportunities for all 
students to show their best efforts. 

Documentation and data collection. Because students’ investigations within the challenging environments described 
above are mediated by technology, many have built-in means for documenting what students are doing as they do it. 
Many of these projects can develop and capture electronic portfolios of students’ investigations as they plan, 
conduct, analyze, revise, interpret, and publish their investigations. Collaborations with peers or telementor/experts 
supported by the technology can also be documented by it. These rich records of students’ inquiry processes in 
action, knowledge acquisition and use, communication, and collaboration skills can form a far more extensive and 
valid database for sampling and analyses of students’ mathematics and science learning. 

Analyses of students’ performance. Analyses of students’ performance can be accomplished automatically with 
computer-generated statistical summaries and tests of quantitative data such as what resources students access, how 
often, and in which ways they access them. A number of projects using CD-ROMs have built-in programs for 
accumulating students’ responses to questions and problems and periodically presenting feedback in the form of , 
charts and graphs to students. The designs embedded in these software programs can provide analyses of students’ 
performance on embedded assessments. Other projects, such as one designed to test domain knowledge, offer 
computer-based scoring of concept map entries ( O’Neill, 1997) . 

Of course, technology can bring to bear its computational power to analyze large sets of students data and generate 
summaries and reports about achievement of performance standards. It can also support statistical studies of the 
effects of types of assessment tasks and alternative versions and accommodations of them on the performance of 
student populations. During pilot tests of assessments, technology can assist in modeling studies of the implications 
of multiple methods for the aggregate and subgroups. 

Technology can support analyses of student achievement by sampling from electronic logs of students’ interactions 
with other students and/or experts and also by organizing students’ work in electronic laboratory notebooks and 
portfolios for qualitative analyses. The electronic artifacts can, in turn, be examined by groups of students, educators, 
and the public for conversations about quality and performance standards. 

Enriched criteria. Despite the wide dissemination of national documents for mathematics and science standards, 
practitioners still struggle with specification of criteria forjudging the quality of students’ performances. Subject 
matter understanding and problemsolving are the most prevalent components of rubrics that are developed to assign 
scores to students’ responses to performance assessment tasks. However, a content analysis of the features referenced 
in such rubrics would reveal quite a range. Few and far between are rubrics aimed at rating the quality of student 
metacognitive strategies. Nor are there many examples of rubrics for assessing collaboration. Some suggestions 
follow (Ouellmalz, 1991) : 

. Innovative technologies, such as the PALS electronic resource library, could support the search for rating 
criteria by those charged to develop rubrics for scoring formal assessments. 

• Electronic resource libraries could collect examples of criteria used to judge students’ performances in 
mathematics and science along with samples of student work and scores or commentary. 

• Pools of criteria developed by classroom teachers could be placed in electronic file cabinets. 

• Resource libraries could offer guidelines that have been used to develop rating criteria. 

Conclusion 

I have argued that technology will continue to make significant contributions to our understanding of science and 
mathematics learning and how our educational systems are preparing students to meet world-class performance 
standards. It can support dialogue about performance standards for mathematics and science learning among all 
constituencies, further understanding of performance standards, provide models and guidelines for developing and 
performance standards, enlarge conceptions about the range of measurable outcomes, and expand available 



methods for designing assessments to test achievement of standards. 

Widely available technologies offer greater support for the performance standards enterprise than is currently being 
used. Models for innovative technology use to advance education reform in mathematics and science exist and are 
under development. We should be continually alert to the ever-expanding potential of technology to support and 
enrich mathematics and science learning. 
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